Gamification, an application of game design elements to non-gaming contexts, is proposed as a way to add engagement in technology-mediated training programs. Yet there is hardly any information on how to adapt game design elements to improve learning outcomes and promote learner engagement. To address this issue, we focus on a popular game design element, competition, and specifically examine the effects of different competitive structures, i.e., whether a person faces a higher-skilled, lower-skilled or equally-skilled competitor, on learning and engagement. We study a gamified training design for databases, where trainees play a trivia-based mini-game with a competitor after each e-training module. Trainees who faced a lower-skilled competitor reported higher self-efficacy beliefs and better learning outcomes, supporting the effect of peer appraisal, a less examined aspect of social cognitive theory. Yet trainees who faced equally-skilled competitors reported higher levels of engagement, supporting the balance principle of flow theory. Our study findings indicate that no one competitive structure can simultaneously address learning and engagement outcomes. The choice of competitive structures depends on the priority of the outcomes in training. Our findings provide one explanation for the mixed findings on the effect of competitive gamification designs in technology mediated training.
Because digital games are fun, engaging, and popular, organizations are attempting to integrate them within organizational activities as serious components, with the anticipation that they can improve employees' motivation and performance. But in order to do so and to obtain the intended outcomes, it is necessary to first obtain an understanding of how different digital game designs impact players' behaviors and emotional responses. Hence, in this study, we address one key element of popular game designs: competition. Using extant research on tournaments and intrinsic motivation, we model competitive games as a skill-based tournament and conduct an experimental study to understand player behaviors and emotional responses under different competition conditions. When players compete with players of similar skill levels, they apply more effort as indicated by more games played and longer duration of play. But when players compete with players of lower skill levels, they report higher levels of enjoyment and lower levels of arousal after game-playing. We discuss the implications for organizations seeking to introduce games premised on competition and provide a framework to guide information system researchers to embark on a study of games.
The implementation of enterprise systems has yielded mixed and unpredictable outcomes in organizations. Although the focus of prior research has been on training and individual self-efficacy as important enablers, we examine the roles that the social network structures of employees, and the organizational units where they work, play in influencing the postimplementation success. Data were gathered across several units within a large organization: immediately after the implementation, six months after the implementation, and one year after the implementation. Social network analysis was used to understand the effects of network structures, and hierarchical linear modeling was used to capture the multilevel effects at unit and individual levels. At the unit level of analysis, we found that centralized structures inhibit implementation success. At the individual level of analysis, employees with high in-degree and betweenness centrality reported high task impact and information quality. We also found a cross-level effect such that central employees in centralized units reported implementation success. This suggests that individual-level success can occur even within a unit structure that is detrimental to unit-level success. Our research has significant implications for the implementation of enterprise systems in large organizations.
Technology-mediated learning methods are widely used by organizations and educational institutions to deliver information technology training. One form of technology-mediated learning, e-learning, in which the platform is the tutor, is quickly becoming the cost-effective solution of choice for many corporations. Unfortunately, the learning outcomes have been very disappointing. E-learning training makes an implicit assumption that learners can apply a high level of self-directed learning to assimilate the training content. In contrast, based on perspectives from social cognitive theory, we propose that instructional strategies need to persuade learners to follow self-regulated learning strategies. We test our ideas with participants who were trained through e-learning to design a website. Our findings indicate that participants who were induced to follow self-regulated learning strategies scored significantly higher on learning outcomes than those who were not persuaded to do so. We discuss our findings, and suggest that the interaction among information technology features, instructional strategies, and psychological learning processes offers a fruitful avenue for future information systems training research.
Although there is substantial research on learning that occurs before adoption of a new information system, there is a dearth of research on postimplementation learning when a new system is assimilated as a routine element of users' work. Hence, during the postimplementation period of a bank's new work flow system, we conducted a longitudinal participant observation study to observe knowledge transfers of users and information technology (IT) professionals assigned to a help desk. We found that although users turned to IT professionals to obtain knowledge related to conceptual understanding and procedures to use the system, they most often turned to other users to obtain knowledge that allowed them to adapt the system to their work. IT professionals, on the other hand, often turned to their colleagues to obtain knowledge that helped them modify the system to emerging innovative uses. These patterns of knowledge transfers can be explained based upon source expertise. Our findings indicate that organizations must sustain designated sources of knowledge such as help desks, but must also establish conduits for users to acquire knowledge from other users and develop innovative uses of the system. A substantial amount of critical knowledge transfers relevant to system adaptation occurred during face-to-face discussions between users and IT professionals, and therefore future research should examine how this would be affected by the outsourcing of technical support functions.
Researchers have emphasized that existing training strategies must be modified in order to adequately prepare users to employ collaborative applications. We utilize findings from the vast amount of training research conducted thus far and point to some problems that might occur when existing strategies are applied to train users of collaborative applications. We test our ideas by conducting a longitudinal field study of a collaborative work flow application. As proposed in a recent knowledge-level framework, our findings indicate that training programs must not solely focus on developing users' system proficiency skills but must also educate users about the business processes that the collaborative application will support. This additional knowledge will enable users to deal with technology-induced changes in the business processes due to the deployment of the collaborative application. Furthermore, we find that training programs should sensitize users to the interdependencies that exist among their tasks and make them aware of the collective consequences of their individual actions. We also found that users have to engage in collective problem solving efforts and continuously learn new knowledge during the process of appropriation of the collaborative application. We propose a training framework that integrates these ideas to prepare users to make effective use of collaborative applications. The proposed framework calls for trainers to be continuously engaged with users and help refine their knowledge during the process of appropriation. We suggest that theoretical foundations rooted in collective learning be adopted to guide training research in collaborative applications.
The resource-based view has been proposed to investigate the impact of information technology (IT) investments on firm performance. Researchers have shown that a firm's ability to effectively leverage its IT investments by developing a strong IT capability can result in improved firm performance. We test the robustness of this approach and examine several related issues. Our results indicate that firms with superior IT capability indeed exhibit superior current and sustained firm performance when compared to average industry performance, even after adjusting for effects of prior firm performance. However, the differences in the results from various analyses suggest that the impact of "halo effects" and prior financial performance of firms must be taken into consideration in future tests of IT capability. Further, it is critical to develop theoretically derived multidimensional measures of IT capability in order to continue to apply the RBV approach to assess the impact of IT investments on firm performance.
Research on training has traditionally viewed errors made by trainees as detrimental to learning. A great deal of effort has been devoted to finding effective ways of preventing errors from occurring during training. Recently, some researchers have adopted a different perspective: that errors may provide a learning opportunity for trainees. What has been investigated less is the specific mechanism through which errors can foster learning. The objective of our research was to investigate and possibly reconcile these differing viewpoints by examining the error recovery process. We found that, in some situations, errors enhance learning when the trainee adopts an error recovery process that emphasizes the goal structure of the task. We suggest several ways of coaching trainees in training sessions to adopt such error recovery strategies.
Users of information technology form mental models that reflect their understanding and knowledge of an information system. These models affect the proficiency with which they use these systems. In this paper, we draw upon assimilation theory of learning to propose and test a two-stage model of mental model development. We examined the effects of two types of training method, namely conceptual model and procedural, and two levels of nature of interaction, namely novel and simple tasks, on end-users' proficiency in forming accurate mental models of an electronic mail system. Our results indicate that the actual mental models of the system formed by the users predict learning success instead of the type of training provided. Subjects who formed mental models that were conceptual in nature performed significantly better than those who formed mental models that were procedural. Main effects for nature of interaction were not significant. However, we observed a significant interaction effect between the models formed by the users and the nature of their interaction with the system. Our findings suggest that end-user performance is enhanced through training methods that provide good conceptual models but only if users form conceptual mental models and retain them.